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(57) An image processing apparatus comprises: an 
image storing unit (4) operable to store an input image; 
a template storing unit (6) operable to store at least one 
template of a body part area; a detecting unit (7) oper- 
able to detect a location of and a size of the body part 
area out of the input image stored in the image storing 
unit (4), by using the at least one template of the body 
part area stored in the template storing unit (6); an or- 
nament information storing unit (5) operable to store or- 
nament information of the ornament having a reference 
point; and an image composition unit (9) operable to 
scale the ornament in accordance with the size of the 
body part area detected by the detecting unit (7), the 
image composition unit (9) operable to locate a refer- 
ence point of the scaled ornament so as to fit with a po- 
sition of the body part area detected by the detecting 
unit (7), and the image composition unit (9) further op- 
erable to compose the scaled ornament and the input 
image stored in the image storing unit (4). 
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Description 

[0001] The present invention relates to an image processing method and image processing apparatus built in an 
image data-treating communication device (for example, a TV phone, a TV conference system, a video mail system, 
5 a video chat system, an intercom, etc.), more particularly relates to an image processing apparatus editing a personal 
image (for example, a facial image of a user, a whole-body image of the user, etc.). 

[0002] There is an amusement apparatus that prints and outputs an image of a user to various kinds of print media. 
The amusement apparatus laps a frame of ornaments (for example, a design, a pattern, etc.) on a personal image to 
compose a composite image, and outputs the composite image to print media (for example, sticker etc.). 
10 [0003] Such amusement apparatus does not have a function to adjust a positional relationship of the personal image 
and the ornament, in itself. Therefore, even when a user arranges the personal image to be settled in an ornament 
frame, the ornament may overlap with the personal image, so that a face part, or a head part, etc. becomes invisible 
and the personal image becomes unclear. 

[0004] Considering the above-mentioned point, reference 1 (published Japanese Patent Application Laid-Open No. 

15 2000-22929) has proposed an art that adds an ornament to a personal image, after adjusting the ornament not to 
overlap with a body part of a personal image, especially with a body part area corresponding to a face and a head. 
[0005] Recently, a camera-built-in cellular phone provided with a function to lap an ornament frame on a personal 
image to compose a composite image, and to transmit the composite image is put in practical use. When a transmitter 
uses such a camera-built-in cellular phone, a receiver can enjoy seeing the image that the transmitter has edited. 

20 [0006] There is a request not to make the ornament not lap over the personal image, but rather positively, to make 
all or a part of the ornament lap over the personal image in an appropriate position and size. 

[0007] Such ornament is an image pertaining to a person; an image that expresses feelings of the person (for ex- 
ample, tears (sadness), wrinkles of a forehead (anger or dissatisfaction), etc.), and an image that expresses personal 
belongings (for example, a hat, glasses, false mustache, a necklace, etc.). 

25 [0008] In such a case, with the prior art, there is a problem that a user has to input a position and a size of an ornament 
that should be arranged, by a keystroke one by one, which results in a very complicated operation. 
[0009] An object of the present invention is to provide an image processing method that can arrange an ornament 
to a personal image in an appropriate position and in an appropriate size, without a complicated operation to be per- 
formed by a user, and an art related thereto. 

30 [0010] A first aspect of the present invention provides an image processing method comprising: establishing an 
inseparable relation between an ornament and arrangement information of the ornament in a body part area; setting 
a location of the body part area in an input image; setting arrangement of the ornament so as to fit with the set location 
of the body part area using the arrangement information related to the ornament; composing the ornament and the 
input image to generate an ornament-arranged output image, and outputting the ornament-arranged output image. 

35 [0011] According to the method described above, an ornament is composed in an appropriate position of the input 
image by inseparably relating the ornament with the arrangement information of the ornament in the body part area. 
Under the present circumstances, a user does not need to specify the position one by one, at which the ornament 
should be placed, but needs only to indicate which ornament is used. Therefore, this method is very easy to operate. 
[001 2] A second aspect of the present invention provides an image processing method as defined in the first aspect, 

to further comprising: setting a size of the body part area in the input image; and fitting the ornament to the input image 
in size, based on the set size of the body part area. 

[0013] According to the method described above, a user does not need to perform sizing of the input image and the 
ornament one by one, therefore, operability can be improved. 

[0014] A third aspect of the present invention provides an image processing method as defined in the first aspect, 
45 wherein the ornament is treated in a form of an image file and the arrangement information of the ornament in the body 
part area is included in attribute information of the image file. 

[0015] According to the method described above, the ornament and the arrangement information related to the or- 
nament can be handled easily. For example, by transferring (for example, downloading) the above-mentioned image 
file from a certain position (for example, from a WEB server) to another position (for example, to a client machine), the 
50 ornament and the arrangement information related with the ornament can be transmitted as one. Upon receiving the 
transferred image file, a user of a destination can immediately perform composition that reflects the arrangement in- 
formation, without indicating the position that the ornament should be placed. 

[0016] A fourth aspect of the present invention provides an image processing method as defined in the third aspect, 
wherein the attribute information is placed in an extended region of the image file. 
55 [0017] According to the method described above, the arrangement information can be stored in one image file, 
without occupying other areas of the image file. 

[0018] A fifth aspect of the present invention provides an image processing method as defined in the first aspect, 
wherein the ornament is treated in a form of an image file and the arrangement information of the ornament in the body 
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part area is included in a name of the image file. 

[0019] According to the method described above, even if the image file is in a format without an extended area, the 
arrangement information can be stored. The image file can store the arrangement information in the file name that the 
image file has originally. As a result, size expansion of the image file can be restrained. 
5 [0020] A sixth aspect of the present invention provides an image processing method as defined in the first aspect, 
wherein the ornament is treated in a f orm of an image file and the arrangement inf ormation of the ornament in the body 
part area is included in another file inseparably related to the image file. 

[0021 ] According to the method described above, the arrangement information can be edited by editing only the file 
that has the arrangement information, without opening the image file. 
10 [0022] A seventh aspect of the present invention provides an image processing method as defined in the first aspect, 
wherein the arrangement information of the ornament in the body part area includes information of an ornament ref- 
erence point. 

[0023] According to the method described above, using the reference point of the ornament, the arrangement of the 

ornament can be expressed briefly. 
15 [0024] An eighth aspect of the present invention provides an image processing method as defined in the second 

aspect, wherein the arrangement information of the ornament in the body part area includes scaling information defining 

a relation between a size of the body part area and a size of the ornament. 

[0025] According to the method described above, a processing in sizing can be simplified. 

[0026] A ninth aspect of the present invention provides an image processing method as defined in the first aspect, 
20 wherein the body part area is a face area of a person photographic object. 

[0027] According to the method described above, an image in which the face area is chosen as the body part area 

becomes more interesting, since the face may vividly expresses the person's intention and feeling. In addition, because 

a reference point is on an ornament, the ornament can be attached in an appropriate position of the face area in an 

appropriate size. For example, an ornament of a hat can be attached in a position of the head of the face area. 
25 [0028] A tenth aspect of the present invention provides an image processing method as defined in the seventh aspect, 

wherein the ornament reference point is one of an upper left comer point, an upper side middle point, an upper right 

comer point, a left side middle point, the central point, the center of gravity, a right side middle point, a lower left comer 

point, a lower side middle point, and a lower right comer point. 

[0029] According to the method described above, arranging the ornament in an input image can be performed ac- 
30 cording to a characteristic of the ornament. 

[0030] An eleventh aspect of the present invention provides an image processing method as defined in the first 
aspect, wherein the ornament is at least one of an image expressing personal feelings and an image expressing 
personal belongings. 

[0031] According to the method described above, there are ways to make an image interesting by adding a feeling 
35 expression to an input image, or by variously changing accessories. In other words, pleasure like a simple fashion 
show can be enjoyed. 

[0032] The above, and other objects, features and advantages of the present invention will become apparent from 
the following description read in conjunction with the accompanying drawings, in which like reference numerals des- 
ignate the same elements. 

40 

Fig. 1 is a functional block diagram, illustrating how an image processing apparatus functions according to the first 
embodiment of the present invention; 

Fig. 2 is a block diagram, illustrating the image processing apparatus according to the first embodiment of the 
present invention; 

45 Rg. 3 is a flowchart, illustrating the image processing apparatus according to the first embodiment of the present 

invention; 

Figs. 4 (a) to (c) are illustrations, showing templates according to the first embodiment of the present invention; 
Fig. 5 is an explanatory diagram, illustrating pattern matching according to the first embodiment of the present 
invention; 

50 Fig. 6 is an illustration, showing a face area detection result according to the first embodiment of the present 

invention; 

Fig. 7 is an explanatory diagram, illustrating each point of a face area according to the first embodiment of the 
present invention; 

Fig. 8 is an illustration, showing an ornament according to the first embodiment of the present invention; 
55 Figs. 9 (a) to (i) are illustrations, showing ornaments according to the first embodiment of the present invention; 

Fig. 10 is an illustration, showing a resultant composite image according to the first embodiment of the present 
invention; 

Fig. 11 is a functional block diagram, illustrating how an image processing apparatus functions according to the 
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second embodiment of the present invention; 

Fig. 12 is a flowchart, illustrating the image processing apparatus according to the second embodiment of the 
present invention; 

Fig. 13 is an illustration, showing a frame image according to the second embodiment of the present invention; 
5 Figs. 14 (a) to (i) are illustrations, showing frame images according to the second embodiment of the present 

invention; and 

Figs. 15 (a) to (c) are illustrations, showing resultant composite images according to the second embodiment of 
the present invention. 

w [0033] Hereinafter, a description is given of the embodiments of the present invention with reference to the accom- 
panying drawings. Prior to detailed description of constructions, important terms are explained. 
[0034] A "personal image" is an image that contains a part or a whole image of a person. Therefore, an image may 
be a whole-body image, a facial image, an image of a sight of a person's back, or the upper half of the body. The image 
may also be a photograph including two or more people. Any kind of patterns, such as scenery and a design other 

*5 than a person, may comprise a background. 

[0035] A "body part" means a part of a person's body. The part of the person's body is included in the body part, 
when the part can be recognized as the part of the person's body, even when the body part is invisible because the 
skin of the body part is equipped with dress, a hat, or shoes. Therefore, a face is a body part and a head is also a body 
part. An eye, a nose, a mouth, an ear, eyebrows, hair, a head, the upper half of the body with dress, a hand, an arm, 

20 a leg, feet with shoes, a head with a hat, eyes with glasses are also body parts. 

[0036] A "body part area" is an area defined as an area which the body part occupies in the personal image. The 
body part area may include, within itself, a part that is not the body part, and may be an area that is located inside the 
body part. 

[0037] Suppose that the body part is a face, the body part area may include an area in the circumference of the face, 
25 or the body part area may include a minimum-sized rectangle surrounding eyes, a mouth, and a nose, but not containing 
a forehead and ears. 

[0038] An "ornament" may be an image pattern to be added. The image pattern may be an image pattern that is 
stored in advance, or may be an image pattern generated with computer graphics technique. The image pattern may 
be a pattern of a character, a pattern of a symbol, and a pattern of a figure. 
30 [0039] "Personal belongings" have a broader sense than a genera] meaning, including clothes, a life supply, goods 
for hobby and amusement, sporting goods, stationery, a small machinery, etc. 

[0040] "Arrangement information" is information determining where to arrange an ornament in a body part area in 
what size, including information regarding a reference point of the ornament and scaling information defining a relation 
between a size of the body part area and a size of the ornament. Here, the ornament may be arranged outside the 
35 body part area. In this case, it is preferable to define a distance between the ornament and the body part area, and to 
include the distance in the "arrangement information." 

[0041] In the present embodiment, the point is that an ornament is indivisibly related with arrangement information 
of the ornament in a body part area. Specifically, there are the following three ways of how to maintain information. 
Any way may be adopted. 

40 

(Example 1) An ornament is treated in a form of an image file, the arrangement information of the ornament in a 
body part area is included in attribute information of the image file, and the attribute information is placed in an 
extended area of the image file. 

45 (Example 2) An ornament is treated in a form of an image file, and the arrangement information of the ornament 

in a body part area is included in a file name of the image file. 

(Example 3) An ornament is handled in a form of an image file, and the arrangement information of the ornament 
in a body part area is included in another file that is indivisibly related to the image file. 

50 

[0042] The above-mentioned (Example 1) to (Example 3) are examples to illustrate. As long as the ornament is 
indivisibly related to the arrangement information of the body part area of the ornament, other optional ways of how to 
maintain information may be adopted. 

[0043] As long as the ornament and the arrangement information are indivisibly related, relationship between the 
55 ornament and the arrangement information may not be limited to only one-to-one correspondence, but the relationship 
may be, for example, one-to-many correspondence or many-to-one correspondence. 

[0044] An ornament does not need to be in a form of an image file, but may be an image pattern outputted by computer 
graphics software and a program, or may be expressed with descriptive language. 
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[0045] Regarding a frame image, the "ornament" is replaced with a "frame image," and "arrangement information of 
the ornament in a body part area" is replaced with "arrangement information of the frame in a frame image," in the 
above-described sentences. 

s (Embodiment 1) 

[0046] Fig. 1 is a functional block diagram, illustrating how an image processing apparatus functions in the first 
embodiment of the present invention. As shown in Fig. 1 , the image processing apparatus of the present embodiment 
has the following components. 

10 [0047] A control unit 1 controls each element shown in Fig. 1 along with a flowchart as shown in Fig. 3. 

[0046] An image input unit 2 obtains an input image. The input image may be a still image and may be an image for 
one frame of a moving image. Furthermore, the input image may be an image immediately after taken with a camera, 
or an image that is decoded from an encoded data of a camera-taken image according to coding methods, such as 
JPEG and MPEG. (The encoded data may be loaded from a recording medium, or may be received from a communi- 

15 cation device.) 

[0049] A display unit 1 0 consists of a display device that displays an input image that is inputted by the image input 
unit 2 and stored in an image storing unit 4, and a composite image that an image composing unit 9 has composed 
an image of an ornament and the input image. 

[0050] An operation unit 3 receives input information from a user. In particular, the user inputs, using the operation 
20 unit 3, information that which ornament information the user uses among a series of the ornament information stored 
in an ornament information storing unit 5. The image storing unit 4 stores an input image that the image input unit 2 
inputs. 

[0051] A template storing unit 6 stores templates of a body part area. Hereafter, in the present embodiment, a face 
area is discussed as an example of the body part area. However, the following discussion is applicable for other body 
25 part areas, such as a hand as well. 

[0052] As shown in Figs. 4 (a), (b), and (c), templates made by each modeled outline of face parts (a head, eyes, a 
nose, and a mouth) are prepared in different sizes for use. 

[0053] In Fig. 1 , a face detecting unit 7 corresponds to the detecting unit described in the claims of the present 
invention. The face detecting unit 7 detects a position of and a size of a body part area from the input image stored in 

30 the image storing unit 4, using the templates stored in the template storing unit 6. 

[0054] Here, as shown in Fig. 5, the face detecting unit 7 extracts an edge component, filtering the input image stored 
in the image storing unit 4 with a differential filter. The face detecting unit 7 selects a template 53 from the template 
storing unit 6, and performs pattern matching using the selected template 53 and the edge component. 
[0055] The pattern matching is a processing which moves the template 53 (Nx * Ny pixels) over a search area (Mx 

35 - Nx + 1) (My - Ny + 1) in an input image (Mx * My pixels) which is larger than the template, and searches the upper 
left position (a, b) of the template at which the residual R given by the following equation becomes minimum. Here, 
the symbol "*" indicates multiplication. 



[0056] In Equation 1 , 1 (a, b) (mx, my) is a partial image of an input image, and T (mx, my) is an image of the template 
53. 

[0057] instead of Equation 1 , the upper left position (a, b) of the template may be searched by calculating a cross- 
50 correlation coefficient C defined by Equation 2, and finding a maximum value for the cross-correlation coefficient C. 
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[Equation 2] 
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[0058] The face detecting unit 7 uses the templates shown in Fig. 4. When the face detecting unit 7 uses Equation 
1 , the face detecting unit 7 finds a template with which the residual R becomes minimum. When the face detecting unit 
35 7 uses Equation 2, the face detecting unit 7 finds a template with which the correlation coefficient C becomes maximum. 
The face detecting unit 7 regards the values of the position and the size of the matched template as the values of the 
position and the size of the face area in the input image. 

[0059] The processing of the face area in the present embodiment is explained using Figs. 6 and 7. Fig. 6 shows an 
example of a detection result of the face area. A rectangular face area 61 is detected from an input image 60. 

40 [0060] In the present embodiment, the face detecting unit 7 determines, using the template mentioned above, a 
coordinate of the upper left comer point, a lateral length L and a longitudinal length M of the rectangular face area 61 . 
[0061] As shown in Fig. 7, in the present embodiment, the face detecting unit 7 determines the positions of nine 
points in total regarding the rectangular face area from the values mentioned above. The nine points are the upper left 
comer point P0, the upper side middle point P1 , the upper right comer point P2, the left side middle point P3, the center 

45 (or the center of gravity) P4, the right side middle point P5, the lower left comer point P6, the lower side middle point 
P7, and the lower right comer point P8. 

[0062] In the present embodiment, a reference point of the ornament as described later is set at one of the above- 
mentioned nine points or at a proportionally allotted point for plural points arbitrarily chosen from the above-mentioned 
nine points. 

so [0063] According to the relationship, the reference point of the ornament becomes an appropriate position of the 
face area, just by adjusting the reference point of the ornament to the set point as mentioned above. 
[0064] Selection of the set point mentioned above is just one example; therefore the point can be arbitrarily changed 
as long as the scope or spirit of the present invention is not changed. 

[0065] When plural faces exist in the image, the face detecting unit 7 detects, as the face part area of a second 
55 person, a position of the lowest residual R or the highest correlation coefficient C within the input image area from 
which the already detected face part area as the first person is excluded. 

[0066] Similarly, for a face of a third person and on, the detecting unit 6 detects repeatedly, as the face part area, 
the low position of the residual R and the high position of the correlation coefficient C in the area repeatedly subtracted 
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for the previously detected face part areas, until the residual R becomes larger than a previously defined threshold or 
until the correlation coefficient C becomes smaller than a previously defined threshold. In this way, face part areas for 
plural persons are detectable. 

[0067] I n the present embodiment 1 , it is desirable to detect not only the face areas, but also each area for face parts 
5 (for example, a right eye, a left eye, a nose, a mouth, both eyes, a right cheek, a left cheek, both cheeks, etc.). This 
point is different from the embodiment 2 that will be described later. Thereby, the ornament can be arranged in more 
detailed manner. 

[0068] As for the face parts, when a template is prepared, a position and a size of the face parts are individually 
detectable in a similar way as the face area. Therefore, detailed explanation is omitted. After detecting only the face 
io area, the size and the position of the face parts may be assumed, for example, by proportionally allotting the position 
and the size of the face part to the position and the size of the face area that is detected. 

[0069] As described above, the face detecting unit 7 determines the position and the size and stores the result into 
the detection result storing unit 8 of Fig. 1 . 

[0070] In Fig. 1 , an ornament information storing unit 5 stores the ornament information. The ornament information 
15 is explained in the following, using Figs. 8 and 9. 

[0071] In the example (an ornament of a hat) shown in Rg. 8, ornament information consists, in a manner related 

with each other, of data of an ornamental image (the picture of the hat), size information (diameter a of the body of the 

hat and diameter b of the brim), the reference point c in the ornament, and the point with which the reference point 

coincides in the rectangular face area. 
20 [0072] Other examples of the ornament include a cap shown in Fig. 9 (a), a headgear shown in Fig. 9 (b), a headband 

shown in Fig. 9 (c), various kinds of eye glasses or sunglasses shown in Figs. 9 (d) to 9 (f), and mustaches shown in 

Figs. 9 (g) to 9 (i). 

[0073] The ornament may be tears, wrinkles between eyebrows, shade of a face, sweat, a mark indicating sunshine, 
and so on, which express feelings of people, or personal belongings defined previously. 
25 [0074] The ornament image data may be a raster image or a vector image. 

[0075] In Fig. 1 , an image composition unit 9 refers to information of the position and the size for the face area that 
is stored in the detection result storing unit 8. 

[0076] The image composition unit 9 scales up or down the size of the ornament that is chosen according to the size 
of the face area, locates the reference point for the ornament so as to fit to the position of the face area, and composes 

30 the scaled ornament and the personal image that is stored in the image storing unit. 

[0077] Locating the reference point and scaling the ornament can be done by easy processing. As for scaling, di- 
rectivity may be given such that scaling is executed only for the lateral direction but not for the longitudinal direction. 
[0078] As for the image composition, a scaled ornament may be arranged in the foreground of the personal image, 
or in the background. A composite image of the scaled ornament and the personal image may be displayed after color 

35 mixing with a suitable alpha value. 

[0079] Processing of the image composition and the ornament scaling is not necessary to be directly performed for 
the personal image and the ornament image, and may be indirectly performed for the personal image and the ornament 
image, using description languages such as an SMIL format, a Shockwave format, etc. 

[0080] The resultant example is shown in Fig. 10, when the ornament of the hat shown in Fig. 8 is applied to the 
40 face area detection result shown in Rg. 6. It should be understood by examining Fig. 1 0 that the hat is scaled to the 
appropriate size (the head size of the person), and is arranged at the appropriated position (the position on top of the 
head, where the forehead is hidden a little bit by the hat). 

[0081] Incidentally, in this kind of the image processing, it may happen to have a result such that a hat that is too 
small to put on the head is arranged in front of a mouth. Such result can not be immediately said an error from the 
45 point of view of the image processing itself; however, it is very unrealistic and inappropriate. 

[0082] Comparing to the unrealistic result mentioned above, the present invention lets a user operate the image 
processing without complicated procedures. The user may choose the ornament of the hat after the personal image 
is inputted, and then the composite image as shown in the Fig. 10 is automatically obtained. Therefore, usability can 
be drastically improved. 

so [0083] Since the ornament and the arrangement information are related to each other inseparably, when the user 
acquires the ornament, the user acquires the arrangement information at the same time. This point also applies to the 
case where the ornament is downloaded from a server and the case where the ornament is retrieved from recording 
media (such as memory card). Therefore, the user needs to only concern about acquiring the ornament and can very 
easily deal with the ornament and the arrangement information. 

55 [0084] Next, an example of detailed construction for the image processing apparatus and the peripheral units shown 
in Fig. 1 is explained using Fig. 2. Fig. 2 is a block diagram of the image processing apparatus mentioned above. 
[0085] In the example of Fig. 2, the image processing apparatus of Fig. 1 is installed in a camera-built-in cellular 
phone. As shown in Fig. 2, the camera-built-in cellular phone has the following elements. 
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[0086] A CPU 21 controls each element of the Fig. 2 via a bus 20 and executes a control program that is stored in 
a ROM 23, following the flowchart of Fig. 3. 

[0087] RAM 22 secures a temporary storage area that the CPU 22 requires for the processing. 
[0088] A flash memory 24 is a device equivalent to the recording medium. 
5 [0089] A communication processing unit 26 performs transmission and reception of data with an external commu- 
nication device via an antenna 25. 

[0090] An image processing unit 27 consists of an encoder/a decoder according to coding methods, such as JPEG 
and MPEG, processes the image (a still image or a moving image) that a camera 28 photographed, or controls the 
display status of an LCD 29 (an example of a display device) based on the image data directed by the CPU 21 . 
10 [0091] An audio processing unit 30 controls the input from a microphone 31 , and the audio output via a speaker 32. 
[0092] The bus 20 is connected to an interface 33, and the user can input operation information by a key set 34 via 
the interface 33. The user can connect other devices via a port 35. 

[0093] A function of the image input unit 2 in Fig. 1 is realized by the process that the CPU 21 or the image processing 
unit 27 performs for data that is stored in the flash memory 24 or for data that the camera 28 photographed. 

15 [0094] Functions of the control unit 1 , the face detecting unit 7 and the image composing unit 9 are realized by the 
process that the CPU 21 performs, by exchanging data with the RAM 22, the flash memory 24 and so on. 
[0095] The image storing unit 4, the template storing unit 6, the ornament information storing unit 5, and the detection 
result storing unit 8 are equivalent to the area secured in the RAM 22, the ROM 23 or the flash memory 24. The key 
set 34 of Fig. 2 is equivalent to the operation unit 3 of Fig. 1 . 

20 [0096] The CPU 21 performs recognition of the operation that the user performs on the key set 34, acquisition of an 
image from the camera 28, compression of the camera image and saving into the flash memory 24, loading and ex- 
tension of the saved image, image composition, image reproduction, and displaying on the LCD 29. The image process- 
ing unit 27 may perform some items of the above-described processing. 

[0097] Next, using Fig. 3, the flow of processing in the image processing apparatus according to the present em- 
25 bodiment is explained. 

[0098] In Step 1 , the control unit 1 controls the image input unit 2 to store the input image in the input image storing 
unit 4 via the image input unit 2 and the control unit 1 . 

[0099] In Step 2, the control unit 1 orders the display unit 10 to display the input image that is stored in the image 
storing unit 4, and then the input image is displayed on the display unit 10. 
30 [0100] In Step 3, the control unit 1 waits for the user to input information that describes which ornament should be 
used. 

[01 01 ] When the user inputs the information by using the operation unit 3, the control unit 1 orders the face detection 
unit 7 to detect the face area in Step 4. Thereby, the face detecting unit 7 detects the position and the size of the face 
area by using the templates that are stored in the template storing unit 6, and stores the detection result into the 

35 detection result storing unit 8 in Step 5. 

[0102] When all the processing mentioned above are completed, the image composing unit 9 scales up or down the 
ornament image according to the size of the face area in Step 6. In Step 7, the image composing unit 9 composes the 
scaled ornament image and the input image so as to locate the reference point of the ornament at the corresponding 
point of the face area, and then stores the composite image in the image storing unit 4. In Step 8, the control unit 1 

40 orders the display unit 1 0 to display the composite image. 

[0103] In Steps from 2 to 5, the face detection may be performed in advance of or at the same time of selecting the 
ornament. 

[0104] In advance of selecting the ornament, if the face detection is started by using time when the user thinks which 
ornament should be used (it is wasteful time for the information processing apparatus), the face detection seems for 
45 the user as if it has completed within the short period of time. 

[0105] In Steps 6 to 7, the ornament image can be scaled after or at the same time of locating the reference point 
for the ornament. 

(Embodiment 2) 

so 

[0106] The embodiment 2 of the present invention is explained using Figs. 11 to 15. Fig. 11 is a functional block 
diagram of the image processing apparatus in the embodiment 2 of the present invention. 

[0107] In Fig. 11 , explanation is omitted by attaching the same symbols regarding the same contents as the embod- 
iment 1 . However, a face detecting unit 7 and a template storing unit 6 differ from the embodiment 1 . The face detecting 
55 unit 7 and the template storing unit 6 need to be applicable only to the face area and the face parts are not necessary 
to be considered. 

[01 08] A frame image storing unit 1 1 stores a frame image with a frame in which a face image is inserted, as shown 
in Fig. 13. A frame with a lateral length d and a longitudinal length e is provided in the frame image shown in Fig. 13. 
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After the size is adjusted, the image of the face area is inserted within the frame. 

[0109] Frame images may include the images shown in Figs. 14 (a) to (i). As shown in Fig. 1 3, "frame" is not limited 
to the one that has visible frame lines, but may include the ones that have invisible frame lines as shown in Figs. 14 
(b), (d), and (e). Furthermore, as shown in Figs. 14 (g) to (i), figures imitating a body without a head are used as the 
5 image within the frame, and the area where the head should exist can be defined as a "frame". 

[0110] In Fig. 11, the face image clipping unit 12 clips only the image of the face area, which the face detecting unit 
7 detects from the input image stored in the image storing unit 4. 

[01 11] The image composition unit 13 scales up or down the image of the face area, which the face image clipping 
unit 12 has clipped, according to the frame size. The image composition unit 1 3 inserts the image into the frame of the 
io frame image and outputs the composite image to the image storing unit 4. 

[0112] Next, the flow of the processing in the image processing apparatus according to the present embodiment is 
explained using Fig. 12. 

[01 13] in Step 11 , the control unit 1 controls the image input unit 2 to store the input image in the image storing unit 
4 via the image input unit 2 and the control unit 1 . 
*5 [0114] In Step 12, the control unit 1 orders the display unit 1 0 to display the input image stored in the image storing 
unit 4, and the input image is displayed on the display unit 10. 

[0115] In Step 13, the control unit 1 waits for the user to input information regarding which frame image should be 
used. 

[01 16] When the user uses the operation unit 3 and inputs the information, the control unit 1 orders the face detecting 

20 element 7 to perform detection for the face area in Step 1 4. 

[0117] Thereby, the face detecting unit 7 detects the position and the size of the face area by using the templates 
stored in the template storing unit 6, and stores the detection result in the detection result storing unit 8. 
[0118] When all the above processing is completed, in Step 15, the face image clipping unit 12 clips only the image 
of the face area from the input image, and outputs the image to the image composition unit 13. 

25 [0119] In Step 1 6, the image composition unit 9 scales up or down the face image that is clipped according to the 
frame size of the frame image. In Step 1 7, the image composition unit 9 attaches the scaled face image to the frame 
and stores the composite image in the image storing unit 4. In Step 1 8, the control unit 1 orders the display unit 10 to 
display the composite image. 

[0120] In Steps 1 5 to 1 7, dipping and scaling the face image may be performed in any order. 
30 [0121] In advance of selecting the ornament, if the face detection is started by using time when the user thinks which 
ornament should be used (it is wasteful time for the information processing apparatus), the face detection seems for 
the user as if it has completed within a short period of time. 

[0122] Fig. 15 (a) shows a resultant example that the face image is attached to the frame image shown in Fig. 13. 
As shown in Fig. 15 (b), plural frames can be provided and the face image can also be attached on each frame. 

55 [0123] In the above discussion, composite image having one face per one frame is explained as an example. As for 
the face detection for a plurality of people, procedures described in the second half of the embodiment 1 can be applied. 
For example, when there are plural faces exist, as shown in Fig. 15 (b), it is preferable to determine a rectangle area 
that surrounds all the face images, to adjust the rectangle area so as to fit to one frame, and to perform the image 
composition. As a result, a plurality of face images is attached within the frame as shown in Fig. 15 (c). 

to [0124] Of course, the embodiment 1 and the embodiment 2 may be combined. 

[0125] According to the present invention, the user can arrange the personal image at a suitable position in suitable 
size without complicated operations. Besides, the user can easily arrange only the face image within the frame of the 
frame image with suitable size. Therefore, the user can edit the personal image easily and interestingly. 
[0126] Having described preferred embodiments of the invention with reference to the accompanying drawings, it is 

45 to be understood that the invention is not limited to those precise embodiments, and that various changes and modi- 
fications may be effected therein by one skilled in the art without departing from the scope or spirit of the invention as 
defined in the appended claims. 



so Claims 

1. An image processing method comprising: 

establishing an inseparable relation between an ornament and arrangement information of the ornament in a 
55 body part area; 

setting a location of the body part area in an input image; 

setting arrangement of the ornament so as to fit with the set location of the body part area using the arrangement 
information related to the ornament; 
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composing the ornament and the input image to generate an ornament-arranged output image, and 
outputting the ornament-arranged output image. 

2. An image processing method as defined in claim 1 , further comprising: 

5 

setting a size of the body part area in the input image; and 

fitting the ornament to the input image in size, based on the set size of the body part area. 

3. An image processing method as defined in claim 1 , wherein the ornament is treated in a form of an image file and 
io the arrangement information of the ornament in the body part area is included in attribute information of the image 

file. 

4. An image processing method as defined in claim 3, wherein the attribute information is placed in an extended 
region of the image file. 

15 

5. An image processing method as defined in claim 1 , wherein the ornament is treated in a form of an image file and 
the arrangement information of the ornament in the body part area is included in a name of the image file. 

6. An image processing method as defined in claim 1 , wherein the ornament is treated in a form of an image file and 
20 the arrangement information of the ornament in the body part area is included in another file inseparably related 

to the image file. 

7. An image processing method as defined in claim 1 , wherein the arrangement information of the ornament in the 
body part area includes information of an ornament reference point. 

25 

8. An image processing method as defined in claim 2, wherein the arrangement information of the ornament in the 
body part area includes scaling information defining a relation between a size of the body part area and a size of 
the ornament. 

30 9. An image processing method as defined in claim 1 , wherein the body part area is a face area of a person photo- 
graphic object. 

10. An image processing method as defined in claim 7, wherein the ornament reference point is one of an upper left 
comer point, an upper side middle point, an upper right comer point, a left side middle point, the central point, the 

35 center of gravity, a right side middle point, a lower left comer point, a lower side middle point, and a lower right 

comer point. 

11 . An image processing method as defined in claim 1 , wherein the ornament is at least one of an image expressing 
personal feelings and an image expressing personal belongings. 

40 

12. An image processing method comprising: 

storing a frame image having a frame to compose a body part area; 
setting a location of and a size of the body part area in an input image; and 
45 outputting a composite image obtained by composing an image of the body part area and the frame of the 

frame image. 

13. An image processing method as defined in daim 12, further comprising fitting the location and size-set image of 
the body part area to the frame in size. 

50 

14. An image processing method as defined in claim 12, wherein the frame image is treated in a form of an image file 
and arrangement information of the frame in the frame image is included in attribute information of the image file. 

15. An image processing method as defined in claim 14, wherein the attribute information is placed in an extended 
55 region of the image file. 

16. An image processing method as defined in claim 15, wherein the frame image is treated in a form of an image file 
and arrangement information of the frame in the frame image is included in a file name of the image file. 
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17. An image processing method as defined in claim 12, wherein the frame image is treated in a form of an image file 
and arrangement information of the frame is included in another file inseparably related to the image file. 

18. An image processing method as defined in claim 12, wherein the arrangement information of the frame in the 
5 frame image includes information of a frame reference point. 

19. An image processing method as defined in claim 13, wherein the arrangement information of the frame in the 
frame image includes magnification information defining a relation between a size of the body part area and a size 
of the ornament. 

10 

20. An image processing method as defined in claim 12, wherein the body part area is a face area of a person pho- 
tographic object. 

21. An image processing method as defined in claim 18, wherein the ornament reference point is one of an upper left 
is comer point, an upper side middle point, an upper right comer point, a left side middle point, the central point, the 

center of gravity, a right side middle point, a lower left comer point, a lower side middle point, and a lower right 
comer point. 

22. An image processing method as defined in claim 12, wherein the ornament is at least one of an image expressing 
20 personal feelings and an image expressing personal belongings. 

23. An image processing apparatus comprising: 

an image storing unit (4) operable to store an input image; 
25 a template storing unit (6) operable to store at least one template of a body part area; 

a detecting unit (7) operable to detect a location of and a size of the body part area out of the input image 
stored in said image storing unit (4), said detecting unit (7) using the at least one template of the body part 
area stored in said template storing unit (6); 

an ornament information storing unit (5) operable to store ornament information of the ornament having a 

30 reference point; and 

an image composition unit (9) operable to scale the ornament in accordance with the size of the body part 
area detected by said detecting unit (7), said image composition unit (9) operable to locate a reference point 
of the scaled ornament so as to fit with a position of the body part area detected by said detecting unit (7), 
and said image composition unit (9) further operable to compose the scaled ornament and the input image 

35 stored in said image storing unit (4). 

24. An image processing apparatus as defined in claim 23, wherein the body part area is a face area of a person 
photographic object. 

40 25. An image processing apparatus as defined in claim 23, wherein the ornament reference point is one of an upper 
left comer point, an upper side middle point, an upper right comer point, a left side middle point, the central point, 
the center of gravity, a right side middle point, a lower left comer point, a lower side middle point, and a lower right 
comer point. 

45 26. An image processing apparatus as defined in claim 23, wherein the ornament is an image expressing personal 
feelings and/or an image expressing personal belongings. 

27. An image processing apparatus comprising: 

so an image storing unit (4) operable to store an input image; 

a template storing unit (6) operable to store at least one template of a face part area; 

a detecting unit (7) operable to detect a location of and a size of the face part area out of the input image 

stored in said image storing unit (4), said detecting unit (7) using the at least one template of the face part 

area stored in said template storing unit (6); 
55 a frame image storing unit (11) operable to store a frame image having a frame into which the face image is 

to be inserted, and 

an image composition unit (1 3) operable to scale an image of the face part detected by said detecting unit (7) 
in accordance with a size of the frame, and said image composition unit (13) further operable to output a 
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composite image after inserting the image of the face part detected by said detecting unit (7) into the frame 
of the frame image. 
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Fig. 1 
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Fig. 3 



GED 



inputting 
image 




t 


displ 
im; 


aying 
ige 



v 


Y 

f 


! detect i ng 
face 


> 




storing 
detection 
result 


> 


f 


sizing 
ornament 


> 


* 


composing 
image 


> 


t 


displaying 
composition 
image 



( °" d ) 



step 1 



step 2 




step 3 



step 4 



step 5 



step 6 



step 7 



step 8 



14 



EP 1424 652 A2 



Fig. 4(a) 
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Fig. 6 
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Fig. 9(a) 
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Fig. 11 
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Fig. 15(a) Fig. 15(b) 
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